Data Filtering for Cluster Analysis by ℓ0\ell_0-Norm Regularization

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted $\ell_0$-norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the $\ell_0$-norm. The convergence of the global minimum points of the approximating problems towards global minimum points of the original problem is stated. The proposed method also exploits a suitable technique to choose the penalty parameter. Numerical results on synthetic and real data sets are finally provided, showing how some existing clustering methods can take advantages from the proposed filtering strategy.Comment: Optimization Letters (2017

An almost cyclic 2-coordinate descent method for singly linearly constrained problems

A block decomposition method is proposed for minimizing a (possibly non-convex) continuously differentiable function subject to one linear equality constraint and simple bounds on the variables. The proposed method iteratively selects a pair of coordinates according to an almost cyclic strategy that does not use first-order information, allowing us not to compute the whole gradient of the objective function during the algorithm. Using first-order search directions to update each pair of coordinates, global convergence to stationary points is established for different choices of the stepsize under an appropriate assumption on the level set. In particular, both inexact and exact line search strategies are analyzed. Further, linear convergence rate is proved under standard additional assumptions. Numerical results are finally provided to show the effectiveness of the proposed method.Comment: Computational Optimization and Application

A decomposition method for lasso problems with zero-sum constraint

In this paper, we consider lasso problems with zero-sum constraint, commonly required for the analysis of compositional data in high-dimensional spaces. A novel algorithm is proposed to solve these problems, combining a tailored active-set technique, to identify the zero variables in the optimal solution, with a 2-coordinate descent scheme. At every iteration, the algorithm chooses between two different strategies: the first one requires to compute the whole gradient of the smooth term of the objective function and is more accurate in the active-set estimate, while the second one only uses partial derivatives and is computationally more efficient. Global convergence to optimal solutions is proved and numerical results are provided on synthetic and real datasets, showing the effectiveness of the proposed method. The software is publicly available

Active-set identification with complexity guarantees of an almost cyclic 2-coordinate descent method with Armijo line search

In this paper, it is established finite active-set identification of an almost cyclic 2-coordinate descent method for problems with one linear coupling constraint and simple bounds. First, general active-set identification results are stated for non-convex objective functions. Then, under convexity and a quadratic growth condition (satisfied by any strongly convex function), complexity results on the number of iterations required to identify the active set are given. In our analysis, a simple Armijo line search is used to compute the stepsize, thus not requiring exact minimizations or additional information

On global minimizers of quadratic functions with cubic regularization

In this paper, we analyze some theoretical properties of the problem of minimizing a quadratic function with a cubic regularization term, arising in many methods for unconstrained and constrained optimization that have been proposed in the last years. First we show that, given any stationary point that is not a global solution, it is possible to compute, in closed form, a new point with a smaller objective function value. Then, we prove that a global minimizer can be obtained by computing a finite number of stationary points. Finally, we extend these results to the case where stationary conditions are approximately satisfied, discussing some possible algorithmic applications.Comment: Optimization Letters (2018

An Active-Set Algorithmic Framework for Non-Convex Optimization Problems over the Simplex

In this paper, we describe a new active-set algorithmic framework for minimizing a non-convex function over the unit simplex. At each iteration, the method makes use of a rule for identifying active variables (i.e., variables that are zero at a stationary point) and specific directions (that we name active-set gradient related directions) satisfying a new "nonorthogonality" type of condition. We prove global convergence to stationary points when using an Armijo line search in the given framework. We further describe three different examples of active-set gradient related directions that guarantee linear convergence rate (under suitable assumptions). Finally, we report numerical experiments showing the effectiveness of the approach.Comment: 29 pages, 3 figure

Total variation based community detection using a nonlinear optimization approach

Maximizing the modularity of a network is a successful tool to identify an important community of nodes. However, this combinatorial optimization problem is known to be NP-complete. Inspired by recent nonlinear modularity eigenvector approaches, we introduce the modularity total variation $TV_Q$ and show that its box-constrained global maximum coincides with the maximum of the original discrete modularity function. Thus we describe a new nonlinear optimization approach to solve the equivalent problem leading to a community detection strategy based on $TV_Q$. The proposed approach relies on the use of a fast first-order method that embeds a tailored active-set strategy. We report extensive numerical comparisons with standard matrix-based approaches and the Generalized RatioDCA approach for nonlinear modularity eigenvectors, showing that our new method compares favourably with state-of-the-art alternatives

Louvain-like Methods for Community Detection in Multi-Layer Networks

In many complex systems, entities interact with each other through complicated patterns that embed different relationships, thus generating networks with multiple levels and/or multiple types of edges. When trying to improve our understanding of those complex networks, it is of paramount importance to explicitly take the multiple layers of connectivity into account in the analysis. In this paper, we focus on detecting community structures in multi-layer networks, i.e., detecting groups of well-connected nodes shared among the layers, a very popular task that poses a lot of interesting questions and challenges. Most of the available algorithms in this context either reduce multi-layer networks to a single-layer network or try to extend algorithms for single-layer networks by using consensus clustering. Those approaches have anyway been criticized lately. They indeed ignore the connections among the different layers, hence giving low accuracy. To overcome these issues, we propose new community detection methods based on tailored Louvain-like strategies that simultaneously handle the multiple layers. We consider the informative case, where all layers show a community structure, and the noisy case, where some layers only add noise to the system. We report experiments on both artificial and real-world networks showing the effectiveness of the proposed strategies.Comment: 16 pages, 4 figure

Learning the Right Layers: a Data-Driven Layer-Aggregation Strategy for Semi-Supervised Learning on Multilayer Graphs

Clustering (or community detection) on multilayer graphs poses several additional complications with respect to standard graphs as different layers may be characterized by different structures and types of information. One of the major challenges is to establish the extent to which each layer contributes to the cluster assignment in order to effectively take advantage of the multilayer structure and improve upon the classification obtained using the individual layers or their union. However, making an informed a-priori assessment about the clustering information content of the layers can be very complicated. In this work, we assume a semi-supervised learning setting, where the class of a small percentage of nodes is initially provided, and we propose a parameter-free Laplacian-regularized model that learns an optimal nonlinear combination of the different layers from the available input labels. The learning algorithm is based on a Frank-Wolfe optimization scheme with inexact gradient, combined with a modified Label Propagation iteration. We provide a detailed convergence analysis of the algorithm and extensive experiments on synthetic and real-world datasets, showing that the proposed method compares favourably with a variety of baselines and outperforms each individual layer when used in isolation

Machine Learning-Based Classification to Disentangle EEG Responses to TMS and Auditory Input

The combination of transcranial magnetic stimulation (TMS) and electroencephalography (EEG) offers an unparalleled opportunity to study cortical physiology by characterizing brain electrical responses to external perturbation, called transcranial-evoked potentials (TEPs). Although these reflect cortical post-synaptic potentials, they can be contaminated by auditory evoked potentials (AEPs) due to the TMS click, which partly show a similar spatial and temporal scalp distribution. Therefore, TEPs and AEPs can be difficult to disentangle by common statistical methods, especially in conditions of suboptimal AEP suppression. In this work, we explored the ability of machine learning algorithms to distinguish TEPs recorded with masking of the TMS click, AEPs and non-masked TEPs in a sample of healthy subjects. Overall, our classifier provided reliable results at the single subject level, even for signals where differences were not shown in previous works. Classification accuracy (CA) was lower at the group level, when different subjects were used for training and test phases, and when three stimulation conditions instead of two were compared. Lastly, CA was higher when average, rather than single-trial TEPs, were used. In conclusion, this proof-of-concept study proposes machine learning as a promising tool to separate pure TEPs from those contaminated by sensory input